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ABSTRACT 



This report describes the efforts of the North Central 
Regional Educational Laboratory (NCREL) during the last several years to 
respond to direct requests from educational stakeholders to help integrate 
data into their decision-making processes related to school improvement. In 
some cases, NCREL cooperated in the development of educational 
decision-making tools, and in others, NCREL began by helping teams of 
practitioners use data to reinforce their curricular and instructional 
decisions. This work has created the platform from which NCREL will launch a 
significant portion of its work over the next 5 years. It has also 
contributed to the development of a portfolio of decision-making tools, 
initially conceived as a "21st Century Evaluation Portfolio" and later 
reframed as a collective Web site of data-driven tools called the "NCREL 
Toolbelt." Some examples are given of NCREL' s efforts in adding value to Tier 
III assessment data, the data collected from standardized, large-scale 
assessments, in Illinois and Wisconsin. Tier I data, ongoing classroom 
assessment data, has been the focus of NCREL efforts in Creve Coeur, 

Illinois. NCREL has also worked on curriculum mapping and the use of the 
General Topic Tracing Map, a data collection tool used in the Third 
International Mathematics and Science study. NCREL' s new signature effort, 
Education Decision Support Systems, has emerged from these efforts, and its 
goal will continue to be making data more available and useful for educators. 
Two appendixes describes the development chronology for two NCREL efforts. 
(Contains 4 figures, 1 table, and 16 references.) ( SLD) 
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Introduction 

As student demographics become more diverse and the gaps between low- and high- 
achieving students grow, it has become increasingly important to make more informed 
educational decisions. This need for better decision making has grown in tandem with the rise in 
standards-based reform and performance-based accountability systems. 

“Data-driven decision-making,” called an educational “mantra” during the 1990s (Nichols & 
Singer, 2000), assumes that some of the scientific methods that solve complex problems in 
medicine and industry can help determine which educational policies, programs, and teaching 
methods are most effective. Well-organized data about how educational systems function, the 
quantity and quality of their inputs, and how students learn can suggest potentially powerful 
solutions to challenges in education. Employing a process to use these data allows stakeholders to 
move beyond arrays of numbers and categories and toward better conceptualizations and more 
informed inquiry. 

Unfortunately, current data about educational inputs, such the qualifications of teachers and 
the rigor of curricula, are lacking, and in most states, data about educational outcomes continue to 
be vague, confusing, and not clearly linked to student learning (McQueen, 2000). Not only do 
educators and policymakers lack quality, accessible data, they also tend to have difficulty 
analyzing and using the data that presently exist (e.g., large-scale standardized test scores) 
(Bernhardt, 1998). Contributing to this problem, educators have had little training in data 
analysis, lack the tools to begin, and perhaps, most important, function in a system where 
decisions are historically based on intuition, philosophy, and retrospect. 

The Call for Data in the NCREL Region 

The recent increases in educational accountability are reflected by increased standardized 
testing in many states, including those in the North Central region. Each state in our 
region, with the exception of Iowa, administers state exams — and Iowa has recently 
legislated specific school accountability requirements. Recently, student performance on 
these large-scale assessments has been linked to consequences; college tuition dollars 
(Michigan), grade promotion and retention (Ohio, Wisconsin), and graduation (Indiana, 
Wisconsin, and, soon, Illinois). The region’s urban districts are also attaching high stakes 
to their assessment programs. As a result, the stakes for student performance are high for 
all participants in the educational system. The appropriateness of this trend is of 
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considerable debate, but few would argue that significant decisions related to school and 
student performance are being made based on student performance data that is gathered 
infrequently and far removed from everyday classroom happenings. 

NCREL has found, through local and national research, issues-scanning (North Central 
Regional Educational Laboratory, 1999), feedback from our partners, and voices from the field, 
that districts and schools throughout our region are searching for better data and better methods to 
report and understand data in order to enhance their ability to make good administrative and 
educational decisions. Schools need tools to see their effects, districts and states need tools to 
deliver resources, and states need tools to convert inaction to action. Each of these tools must be 
fueled by data. 

This report will describe NCREL ’s efforts during the last several years to respond to direct 
requests from educational stakeholders throughout our region to help integrate data into their 
decision-making processes related to school improvement. In some cases, our response was to 
cooperate in the development of educational decision-making tools. In others, NCREL began by 
helping teams of practitioners use data to inform their curricular and instructional decisions. It 
then evaluated the impact of these interventions and used what was learned to coordinate the 
development of online decision-making tools. Overall, this work has contributed to NCREL s 
growing expertise in the research and development of education decision support systems and has 
created the platform from which we will launch a significant portion of NCREL’s work over the 
next five years. It has also contributed heavily to the development of a portfolio of decision- 
making tools, initially conceived as a “21 st Century Evaluation Portfolio,” and eventually 
reframed as a collective Web site of data-driven tools entitled the “NCREL Toolbelt.” First, some 
background information is needed to understand how and why NCREL came to focus on this 
work. 

NCREL’s Response to the Call for Data 

Technology has always been a central component of NCREL’s efforts to make schools more 
productive. Almost a decade ago, NCREL was an early adopter of gopher and other embryonic 
Internet information retrieval tools. Our Pathways school development library was one of the first 
resources available to schools as the Internet moved from command prompts to graphical 
interfaces and platform-independent browsers. 

While data are not dependent on technology, technology provides a support that can make 
data much more useful. At the same time that Pathways was being developed, NCREL, in 
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collaboration with the Council of Chief State School Officers, initiated an effort to collect 
detailed systematic data on state assessment programs. Using Internet-based survey forms and 
data collection processes, the State Student Assessment Program (SSAP) database was created. 
The database was the first to provide detailed, accurate information about the scope and content 
of the assessment programs in each of the 50 states. 1 

While building this knowledge base of assessment practice, we were also talking with school 
and district personnel about how useful assessment and other data were to their decision making 
process. The resulting conversations suggested that practitioners found themselves in a less-than- 
supportive data environment. 

At one regional conference in the early 1990s, a school superintendent in Illinois hosted a 
meeting with his colleagues. He asked the two dozen superintendents in the room to fold their 
arms on the conference table, place their heads on their arms, and close their eyes. After some 30 
seconds, he explained why they were in silence, in darkness, and alone. He wanted to simulate the 
decision-making context: alone, without support; with little guidance or firm research; and with 
no data. 

Two states away, in Bloomington, Minnesota, a curriculum director was generating grant 
support to build data systems to support his teachers. “Linkages across Minnesota” connected 
teachers in nine school districts electronically and in person. Technology tools to build and share 
lessons aligned to state standards were developed with private partners including Norris 
Educational Innovations, Inc. These tools were extended to link local student performance 
measures to state measures and to test which instructional interventions most closely aligned with 
student growth. 

“Linkages across Minnesota” placed teacher and student at the center of the educational 
process. Personalized road maps for each student’s intellectual journey were to be drawn in an 
effort to meet the student’s instructional needs. 

In 1995 Illinois introduced a new set of standards and a new mandated school improvement 
process. The state desired software to take a lead role in supporting schools to meet the new 
mandates, arguing that that was more efficient than hiring numerous consultants and support 
personnel. It turned to NCREL to help design and build software that would permit schools to 
collect, manipulate, and represent data for use in school improvement decisions and reporting. 
The ISIP software was distributed to almost 4,000 schools. Some 400 schools installed and tried 
ISIP; 40 schools used it long term. The barrier to ongoing use of ISIP, it was found, was not 
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related to the system or the software. The barrier was human. That is, without direct personal 
contact, very few schools had time or initiative to experiment with a new way of doing things. 

These three examples, along with many others, suggested to NCREL that decision support 
systems needed to support decisions by teachers and students and administrators, not just about 
them. In addition, they informed us that tools with technological underpinnings were not 
sufficient. Tools also needed to fit into an ongoing social enterprise. Tools that could not be 
grasped would not be used, no matter how shiny, strong, or profound. Finally, they highlighted 
that the data underlying decision support tools should measure outcomes that are understood and 
valued at the local level. That is, if data were to change people’s practice, the data needed to 
speak to the practices themselves, not some distant construct. 

User-Friendly Data 

Data set up tensions between what is believed and what is seen and can signal the need for 
change. Data also suggest the means by which the results of interventions can be measured and 
tracked. Yet data must match the needs of the user. Stakeholders throughout the educational 
system have distinct decision-making needs and thus require different types of information. In 
addition, a barrage of data can be overwhelming. Data must be organized or tailored for specific 
audiences. 

One way to conceptualize the broad array of data that are available to educators is to place 
them into three categories: Assessment, Demographic, and Program 2 (see Figure 1). Each of these 
categories is described below: 
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Figure 1, Examples of Common Educational Data Sources 




Assessment Assessment may be defined as any method used to better understand the 
current knowledge that a student possesses (Dietel, Herman, & Knuth, 1991). Current 
knowledge” implies that what a student knows is always changing; to make judgments about 
student learning, outcome data on equivalent assessments should be tracked over time. 

A multitude of assessments are available to educators. For the sake of simplicity, assessment 
data in Figure 1 are represented in three tiers according to their purposes and the type of feedback 
they provide. These tiers are described below in Table 1: 
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Table 1. Tiers of Assessment 





Assessment 

purpose 


Rate of 
feedback 


Type of 
feedback 


Primary target 
of feedback 


Tier I 


Annual large- 


Infrequent 


General, broad 


General 




scale 


i 


L 


j 


k 


accountability 














audience: 














Policymakers, 














Community, 














Administrators, 














etc. 


Tier II 


Periodic Grade 










Administrators, 




Level/Subject 










Teachers 




Area 


i 


f 








Tier HI 


Ongoing 


1 


f 


1 


f 


Teachers, 




Classroom 


Frequent 


Specific, narrow 


Students 



Demographic. Demographic data help educators understand the population of students, 
parents, and community members that is served by their school or district. Demographic data do 
not explain performance and are not within the control of educators. However, demographic data 
may help educators to target interventions — individual schools and districts often look at 
performance differentials within their own populations and consider how to overcome them. 

Program. Data about educational programming, such as the scope and sequence of the 
curriculum, the qualifications of teachers, student participation in extracurricular activities, and so 
on can suggest overall school trends (e.g., over the past five years, there has been a growth in the 
percentage of seniors taking advanced placement courses) or needs (e.g., next year, 60 students 
will qualify to take Algebra I in eighth grade, but none of our current teachers are certified to 
teach this course). Unlike demographic data, program data is within the control of schools. 
Schools have the power to improve, add, or terminate programs. These data may be linked to 
student assessment data to find relationships that can, in turn, suggest the need for further, more 
specific program evaluation. 

Although these categories do not comprehensively capture the universe of educational data 
(e.g., financial and perception data are not represented), they contribute significantly to the 
overall picture of the education system’s performance. Each of these data sources provides a lens 
on what transpires each day in school and can clarify trends and effects that are easily missed. 
When used together, they can address the information needs of a variety of educational 
stakeholders. 
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Like the data that fuels them, data tools must suit the needs of schools and other educational 
agencies. Most educators and policymakers are not statisticians. They need tools and resources 
that can help them see better and that provide meaningful information quickly, reliably, and 
cheaply. Too frequently, administrators receive a ream of outcome data from a mandated 
assessment, skim the overall results, and set the remainder of the report on a shelf. Other 
practitioners express a strong desire to better understand these data, matched only by frustration 
at their lack of training in this area. Data tools are necessary to distill information and to link 
stakeholders to results and solutions. When educators can draw inferences from their data, they 
cannot only see the need for change, but can identify the direction of change needed, pinpoint the 
students needing intervention, and identify programs offering promising solutions. 

In business and government arenas, Decision Support Systems (DSSs) have become popular 
and increasingly sophisticated solutions to the need for data tools. 3 Current conventional wisdom 
suggests that DSSs should be automated, computerized systems that collect data into a database 
where they reside until the moment they are brought out in a timely, valid, and understandable 
report. Nationwide, the development of DSSs and Web-based reporting tools for educational 
decision makers is increasing. But SEAs and other educational agencies face barriers to the 
development of DSSs, including: 

• Financing to purchase and maintain modem technology 

• Identifying and retaining staff with the requisite technical skills 

• Finding vendors of commercial DSS products that understand schools and education 

• Attempting to model decisions made in schools, which tend to be inconsistent, highly 
personal, and illusive to capture on a flow chart. 

In spite of these difficulties, many educational agencies have developed, or are in the process 
of developing DSSs. Thus far, four SEAs in the Midwest, including Illinois, Wisconsin, and 
recently, Indiana, and Minnesota, have looked to NCREL for assistance in these projects. 

Adding Value to Tier III Assessment Data 

Many practitioners and researchers, with reason, cite the severe limitations of using Tier HI, 
or large-scale, standardized assessment data for decision making. The data resulting from these 
assessments gives a broad view of school effectiveness. It offers little guidance to teachers 
seeking to make daily instructional decisions and design individualized educational interventions. 
The data are also inefficient indicators of student progress over time. However, these data can be 
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made more meaningful when they (1) can be rapidly disaggregated, (2) can be couched within 
longitudinal trends, and (3) can serve as a basis of comparison with other, similar schools. 
NCREL used these three elements to guide the development of the following state-level decision 
support systems: 

Illinois School Improvement (ILSI) (http://ilsi.isbe.net/ ) 

In June 1999, the Illinois State Board of Education (ISBE) requested help in developing a 
DSS that would assist Illinois educators analyzing and making use of state testing data. Two 
major functionality goals were identified: 

• Design a user-friendly computer software system that enables schools to identify similar 
schools and access useful data on these schools. 

• Design a user-friendly electronic reporting system that enables schools to merge state 
assessment data with their classroom, school, and district data and to print reports that are 
useful to various audiences 

NCREL responded to this call, agreeing to codevelop a school improvement Web site for 
ISBE (goal 1) that was question-driven and would later serve as a template for similar resources 
in other NCREL states. The site would be modeled after the school improvement site in Maryland 
('http://www.mdkl 2.org/) . which was designed to meet many similar goals. Besides being 
interactive, the Maryland site has a graphical approach to data analysis that could be joined to 
school improvement decision making and resources for change. NCREL chose to subcontract 
with the development and design teams (Sligo Computer Services, Inc. and UAQA) that had 
worked on the Maryland. Because of this, the Illinois school improvement Web site was at times 
referred to as a “daughter” of the successful Maryland site. 

The Illinois School Improvement (ILSI) Web site incorporated suggestions from the 
partnership and from a pilot group representing practitioners throughout the state. 4 The site was 
broadly organized into four sections, each a complementary component of the school 
improvement process and each tied to general, but important questions that can drive effective 
inquiry. These sections include: 

• Standards: Where do we need to be? 

Presents the Illinois Learning Standards for elementary and high school students and 
provides resources on how to implement the standards in the classroom. 

• Analysis: How are we doing? 
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Provides data-based reports on an individual school’s achievement against the Illinois 
Learning Standards and presents comparisons: 

■ Of a school to its district and the state averages 

■ Of progress over time 

■ Of a school to other schools with similar student populations. 

• Process: How will we get there? 

Provides background information about the school improvement process and includes an 
interactive school improvement “starter kit.” 

• Knowledge: Where can we find resources? 

Provides information aligned to the Illinois Learning Standards organized by subject 
matter and key topics. Resources include: 

■ Standards-related material 

■ Model programs 

■ Model schools 

■ Professional development 

■ Research. 

QSP for Illinois 

In addition to ILSI, NCREL agreed to partner with the National Center for Research on 
Evaluation, Standards, and Student Testing (CRESST) to customize their Quality School 
Portfolio software (QSP) 5 for Illinois. QSP software operates as a “data manager” that allows 
educators to import and store multiple databases, then acts as a tool to group, view, and report the 
data within these files (National Center for Research on Evaluation, Standards, and Student 
Testing, 1999). Schools enter their own data, set their own goals, and are able to create student 
groupings that respond to their local, specific questions. QSP accepts data that are categorical, 
numerical, textual (string), and chronological (i.e., dates). QSP also delivers a Resource Kit that 
helps schools assemble data. 

The Illinois version of QSP was proposed to meet the local data needs of schools as stated in 
goal 2. Illinois schools would be able to house, analyze, and integrate more local data, including 
assessment data from Tier II and more specific demographic and program data than would be 
available for public viewing on the ISBE Web-based DSS. It was hoped that with these two 
complementary DSSs (i.e., ILSI and QSP), Tier m data would become easier to use and more 
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valuable to Illinois educators, and could be coupled locally with more specific data, including 
data from Tier II assessments (See Figure 2). 

Figure 2. Broad and Local Data Focuses of Illinois DSSs. 



Wisconsin Network for Successful Schools 

(WINSS) (http://www.dpi.state.wi.us/sig/index.html) 

With the development of ILSI under way, Wisconsin soon prioritized the need for its own 
DSS. Wisconsin’s governor asked the Department of Public Instruction (WI DPI) to create a 
Web-based electronic reporting system on Wisconsin school performance, and offered a small 
stipend to begin this work. The objectives for this project were to: 

• Provide educators with online and easily accessible data and graphic illustrations of local 

and statewide results from state assessments for comparative purposes 

( 

• Help schools analyze their state assessment data and guide them in making data-driven 
instructional decisions that support improved performance for all students 

• Facilitate reporting capabilities across schools and districts, and statewide. 

In January 2000, Wisconsin turned to NCREL for assistance. NCREL offered to provide 
technical support to the process of determining the content framework and data analysis 
procedures on the Web site, to assist in the adaptation of the Illinois Web site to the needs and 
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vision of Wisconsin, and to provide substantial funding for the development of a fully functioning 
Web site. 6 

Although unmistakably spawned from the Maryland site and showing familial resemblance to 
ILSI, the Wisconsin Information Network for Successful Schools (WINSS) still exhibits its own 
unique personality. Namely, the mission of Wisconsin educators includes helping students strive 
towards academic and behavioral objectives. This is evident throughout the framework, which is 
summarized below: 

• Standards and Assessment: 

Outlines Wisconsin’s expectations for students. Provides ideas, information, and tools to 
assess progress. 

■ What are our academic standards? 

■ What are standards of the heart? 

■ What indicates success in academic standards? 

■ What indicates success in standards of the heart? 

• Data Analysis: 

Allows users to analyze data about a school by typing in the school or school district 
name. This directs users to the following question-driven analyses: 

■ How are students performing academically? 

■ What programs, staff, and money are available? 

■ What about attendance and behavior? 

■ What are student demographics? 

The site responds to questions by displaying school-specific graphs and tables to help 
users make use of these data. 

• Continuous School Improvement: 

Describes the characteristics of a successful school as well as ideas and tools to help 
school communities in the improvement process. 

■ What makes a successful school? 

■ How can we improve? 

■ Who should be involved in improvement? 

■ Where can we find planning tools? 

• Best Practices: 

Helps answer key questions related to implementing each of the seven characteristics of a 
successful school. 
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• Vision 

• Evidence of Success 

• High academic standards 

• Family, school, and community partnerships 

• Standards of the heart 

• Professional development 

• Leadership 

Wisconsin decided against customizing QSP for the state as part of this effort and tabled 
plans for local DSSs until the roll-out of WINSS. 

The Partnerships 

The partnerships that were forged between NCREL and the states were key to the 
development of the DSSs. In both states, NCREL provided technical assistance and financial 
support to build these DSSs. Specifically, because these projects were consistent with NCREL 
work being funded by federal resources, the laboratory agreed to integrate the cost and personnel 
time needed to codevelop the tools into ongoing budget and staffing plans. In turn, each state 
agreed to: 

• Provide leadership in customizing the DSS to the state’s needs. 

• Assign staff with appropriate expertise to work toward successful completion of the DSS. 

• Provide data and content for the DSS. 

• Provide and implement related professional development to districts and schools. 

• Create a budget and plan to maintain and update the DSS once NCREL funding has been 
expended. 

In Illinois, the partnership was rounded out by the Illinois Business Roundtable (IBRT). The 
IBRT Director of Education Policy and Initiatives contributed greatly to project management 
needs and provided a link to the formative feedback and positive public relations offered by 
Illinois’ industry leaders. In addition, CRESST worked closely with the Illinois partnership to 
tailor QSP for their needs. 

In Wisconsin, the partnership included the Governor’s Office, which provided some financial 
assistance for the site, offered formative feedback during the development process, and aided 
plans for a statewide roll-out. 
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Evaluative Feedback on ILSI, IL QSP, and WINSS 

Information will be needed on how these DSSs are being used and their impact on decisions 
at the systemic and individual student level. To this end, NCREL has planned to design and 
implement formal evaluations of ILSI, IL QSP, and WINSS during the next several years and will 
use the resulting information to continuously improve these and other similar DSSs being 
developed within the NCREL region. 

In the meantime, NCREL has received preliminary feedback on how WINSS and ILSI have 
been used. Between its September 20 launch and October 10, there were over a quarter million 
hits on WINSS, or about 100,000 page views. By December 31, 2000, there were over 1 million 
hits. The ILSI Web site received approximately 100,000 hits within two weeks of its public roll- 
out, and approximately 150,000 hits within the first month. The data analysis features of the ILSI 
site were used most frequently, and users spent, on average, 20 minutes at the site. 7 This 
information suggests a receptive audience for integrating data into the school improvement 
process and a willingness to experiment with new methods for data analysis. Information related 
to use of WINSS, ILSI, and related sites will be formally tracked during the next contract period. 

NCREL also presented the ILSI and WINSS sites at one of four regional “Education Decision 
Support Systems Meetings” hosted by the U.S. Department of Education, Office of the Chief 
Information Officer (OCIO) and the Council of Chief State School Officers (CCSSO). 
Discussions took place on the features and functionality of ILSI and WINSS as they compared to 
other educational DSSs in place or under development nationwide. The 16 DSSs that were 
presented during the meetings were grouped into four general models 8 : 

• Big Systems include individual records in a data warehouse and use sophisticated 
database management and Web tools to manage and generate reports. These systems are 
complex and relatively expensive. 

• Big Pictures look at extant statistics stored as aggregate records in a data mart. These 
systems allow users to see the big picture but are limited in the details that can be 
analyzed and reported because they are built with aggregate records. 

• Big Ideas begin with questions (and possibly decisions to be made) and information 
needs that drive analyses. A data mart and/or data warehouse can be built from which to 
generate custom reports to match the questions. This model is called big ideas because 
the impetus for the system and for the specific reports coming from it originates from 
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individuals’ ideas or information needs. These systems grow in spurts as new needs are 
identified. 

■ Big Family refers to the characteristic of having everyone from the classroom to the SEA 
linked into the same system. The system performs daily transactions (attendance 
accounting, grade reporting, financial accounting, and so on) on servers that allow the 
data to be shared by all levels of the education enterprise. 

ILSI and WINSS are both included in the Big Pictures model of DSSs. Like most of the other 
16 DSSs, ILSI and WINSS are Web-based, focus analyses on assessment results and other state 
accountability measures, and able to represent longitudinal trends in the data. Half of the 
presented DSSs do not go beyond reporting the results of basic descriptive and cross-tabulation 
analyses. ILSI and WINSS do by providing the opportunity to observe correlations between 
student assessment outcomes and various student demographic variables (e.g., SES, ethnicity, 
gender, and so on). 

The most common risk factors noted among the DSSs included site maintenance, continued 
funding, and data quality. For ILSI, other risk factors were noted, such as the complexity of the 
site, the acceptance and use of the site by stakeholders, and the availability of data for the site. 
This initial feedback provides valuable guidance for NCREL’s design of future evaluations for 
these tools. 



Focus On Tier I: Classroom-Level Data 

Tier I data, or ongoing classroom assessment data, are at the heart of the educational process. 
These data reflect the interactive activities of teaching and learning, the so-called “black box” 
between system inputs and outputs (Black & Wiliam, 1998). Although teaching and learning 
ultimately drive reform, reformers often ignore the sufficient measurement and analysis of these 
activities; moreover, these data are seldom represented in educational DSSs. 

Over the past decade, NCREL has invested time and resources into exploring the use of Tier I 
data for ongoing, formative evaluation of the learning process and its effect on student 
achievement and system functioning. Based on conversations with colleagues inside and outside 
the lab, NCREL’s researchers found and visited schools across the U.S. that collected data from 
the teaching and learning process daily, even hourly, and unobtrusively. These schools’ processes 
for generating data were closely studied. More critically, the ways these schools used data proved 
highly informative. Often, it was the student who was the most frequent user of his or her 
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learning data, not the administrators. Teachers embodied the spirit of the scientist-practitioner. 
Frequent, collaborative data analysis helped them understand how the teaching and learning 
process was affecting the progress of each child. The data were typically treated formatively, 
experimentally, in a “what-if-I-do-this-next” mode. This approach was distinct from the 
summative approach more commonly found in schools. These schools used data more to help 
make decisions, and much less to validate decisions already made. 

NCREL used these findings to drive the development of the Practitioners ’ Collaborator y: six 
small Chicago neighborhood schools funded in 1995 by the Chicago Annenberg Challenge. 

These schools proposed to build a laboratory and a collaboration. The purpose of the laboratory 
was to test classroom instructional techniques to identify those that worked and to find the 
courage to discard those that did not. Applying what had been learned about using data in 
schools, the group chose to reflect collaboratively about their practices and their results. Their 
vision was that together, teachers would become more like laboratory scientists, specifying 
outcomes, measuring results, testing hypotheses of increasing subtlety and strength and learning 
from their own and their common practices. Although encouraging, much of this work emerged 
in unique educational settings and with distinctive student populations. The ability to scale up 
these techniques needed to be evaluated in a public school setting. This opportunity presented 
itself with a request from the Creve Coeur School District. 

The Need in Creve Coeur 

In 1998, the superintendent of Illinois District 76, Creve Coeur, contacted NCREL to ask for 
assistance in the district’s movement toward reform student learning. Creve Coeur is located on 
the outskirts of Peoria and has the following characteristics: 

• Population: ~ 6000 

• Enrollment: ~ 820 

• # Schools: 3 (preK to 8) 

• Demographics: 96% Caucasian 

45% free-reduced lunch (state avg. = 36%) 

35% “non-traditional” homes 
Mobility rate: 22 - 38% 

Expenditure/student: $3921, 36% lower than Illinois average 
Designated by Illinois as an “Economically-Challenged School 
District”Despite the implementation of various school reform initiatives over 
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the years, the Creve Coeur district had a history of mediocre student performance on standardized 
tests such as the Illinois Goal Assessment Program (IGAP). 

Previous initiatives in Creve Coeur had emphasized the use of data to guide decisions related 
to school reform. For example, the curriculum director had performed gap analyses between the 
district and state curriculum for each grade and had facilitated work sessions with teachers to 
review this information. Teachers had worked together to better align the content and timing of 
their curriculum to the state standards and to maintain alignment within the local curriculum from 
Grades K through 8. Data from the state standardized test had been disaggregated and distilled in 
reports for administrators and teachers. Each week, data were also collected within classrooms on 
knowledge of math facts throughout the district, but these data were not analyzed or reviewed on 
a regular basis. 

The need for improved student learning in Creve Coeur, along with the district’s inclination 
to use data to drive their work, created an opportunity to improve the district’s focus on high- 
periodicity (i.e., frequently occurring), classroom-level data in an effort to inform the day-to-day 
decision making inherent in an educational setting. A partnership between NCREL, Illinois 
Institute of Technology (IIT), and the Creve Coeur school district was established. The objectives 
of the partnership included working with a group of teachers to build a data-driven model of 
instruction. Specifically, the partnership aimed to co-develop a classroom procedure that would 
capture ongoing assessment data and feed it back to teachers for immediate evaluation on the 
effectiveness of instruction. Work began in the first quarter of 1999 and continued until the 
second quarter of 2000. 

Precision Teaching: A "Hands-On,” Classroom-Based DSS. Six teachers, selected by 
their administrators, met with NCREL/UT staff over the course of approximately five months 
during the spring semester of 1999. Initially, the meetings focused on the importance of using 
formative data to inform classroom decision making and also served to assess techniques and 
tools that might best meet the teachers’ needs. The partnership also prompted teachers to define a 
focus for their improvement efforts. 

The partnership noted that students were having particular difficulty in math as reflected by 
standardized test scores and teacher observations. From informal conversations and by examining 
their curriculum, the teachers realized that year after year, they were spending a large portion of 
their instructional time on reviewing and reteaching basic math skills. They hypothesized that 
their students lacked fluency (i.e., speed and accuracy) on basic math skills, and thus appeared to 
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be stalled in their attempts to master higher-level mathematical skills. It was decided that the 
group would focus their improvement efforts primarily on mathematics. 

Teachers were introduced to a process NCREL and ITT had observed being used by “data 
rich” schools (e.g., Ben Bronz Academy, Momingside Academy) with great success. The process, 
called Precision Teaching (PT) (Lindsley, 1990) began more than 30 years ago with the invention 
of the Change Chart (see one version in Figure 3), a tool for visualizing and making decisions 
about learning and performance. In essence, the Change Chart acts as a hands-on, paper and 
pencil decision support system for precision teaching and learning. 

The use of PT is supported by substantial research (Bushell & Baer, 1994; Fuchs, Fuchs, 
Hamlett, & Stecker, 1991; Johnson & Layng, 1994; Lindsley, 1992) related to using student data 
to guide instruction by pinpointing specific learning goals, measuring student performance related 
to these goals, visually displaying these data to highlight individual student learning trends, and 
then basing the next instructional steps on each student’s particular needs. It is a particularly 
effective method for helping learners obtain fluency on basic skills. The Creve Coeur teachers 
and administrators agreed to apply the decision support process used in PT within their classes in 
an effort to meet their math goals. 

NCREL/IIT staff trained teachers to pinpoint measurable academic performance goals, 
manage daily practice sessions, monitor student charts, and make instructional decisions based on 
Change Charts displaying individual student data. Students were trained to chart and share their 
data with others and to become active participants in the learning process. This included: 

• Setting their own learning goals (“aims”). 

• Analyzing their data to determine the speed and direction of their learning progress. 

• Advocating for themselves by asking for help from a teacher or peer if their data showed 
they were not learning. 

• Asking for more challenging work if they reached their goals. 

On a daily basis, students had timed practice sessions on math facts to measure their fluency 
(i.e., speed and accuracy). Students recorded their performances on the Change Charts (number 
correct and incorrect) and teachers worked with them to adjust their goals, the curriculum, or the 
instruction based on how well students were improving and learning. Weekly, students displayed 
their charts and presented their learning data to the rest of the class. NCREL/IIT staff periodically 
observed teachers and students during these practice and data-sharing sessions, offering 
immediate feedback and suggestions on the process, encouraging data-based decisions, and 
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responding to specific questions. In addition, NCREL/HT staff facilitated teacher-only data- 
sharing sessions after school. 

Figure 3. Example of Change Chart used to track daily progress in Precision Teaching. The 
chart shows student learning progress over time. 




A four-day institute took place during July 1999 to plan for an improved implementation 
during the fall semester. Administrators were trained to manage the work of teachers and to 
provide instructional leadership by attending student chart-sharing sessions once a week. The 
administrators also agreed to collect data on student learning trends within each classroom, to use 
these data to guide interventions with teachers, and to report their findings to the superintendent. 
Teachers discussed their needs to improve PT methods through improved organization, flexible 
student grouping, and peer coaching. Teachers reported that they were unaccustomed to making 
frequent, individualized decisions about student learning; at times, this made instructional 
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decision making difficult and stressful. To improve this situation, they agreed that their math 
materials should allow students to move at an individualized pace through the curriculum. 

After investigating several options, Creve Coeur teachers chose to use basic-skills materials 
(i.e., Momingside Math (Johnson, 1996)) that had been specifically designed to progress in small, 
sequenced curricular steps based upon individual student performance. Then they decided to 
compact the basic-skills component of the math curriculum into a daily period of approximately 
twenty minutes, which would consist of timed, individualized practice, data collection and 
sharing, and instructional decision making. After each practice session, traditional didactic math 
instruction would take place. 

Evaluation. The partnership worked together to develop an evaluation plan for the 
1999/2000 academic year that would describe how frequent, classroom-based data-driven 
decision making affected student performance on various outcome measures. 9 The evaluation 
plan included continued consultation and monitoring by the NCREL/IIT partners as well as 
significant data collection and peer consultation by teachers. Outcome measures were chosen to 
speak to a variety of stakeholders within the district (i.e., Change Charts, curriculum-based tests, 
achievement tests). In addition, qualitative data were collected in the form of classroom 
observations, field notes, and teacher tape-recorded journals. 

Summary of Quantitative Results. Through the end of February 2000, results strongly supported 
the proposition that the implementation of frequent, data-driven decision making in classrooms as 
embodied in the PT decision support system, complemented by Momingside math materials, had 
a robust positive impact on student achievement as measured by timings on basic mathematic 
skills (i.e., addition, subtraction, multiplication, division). All of the overall comparisons showed 
that the data-driven, PT classes ended up with superior performance to a statistically significant 
degree. While this finding was particularly strong in Grades 1,3, and 4, it must be noted that 
within-grade comparisons were mixed; not all PT classes outperformed all “control” classes on 
all skills. Furthermore, the results may be understatements of the power of data-driven decision 
making in the district for three reasons: 

• First, it was discovered that teachers in the control classrooms had their students practice the 
outcome measures once per week, while teachers in the PT classrooms used the outcome 
measures with students only at evaluation times (Time 1, September; Time 2, December; 
Time 3, February; Time 4, May). A potential “practice effect” may have artificially inflated 
the results for the control classes. 




The Call for Data-Driven Decision Making in the Midwest’s Schools— 19 



24 



• Second, analyses may be artificially biased toward the control classrooms due to the larger 
number of participants in these groups. 

• Third, several of the classes were unable to maintain the full implementation for the entire 
year. 

Between Time 3 and Time 4, results varied and sometimes diverted from the existing trends. 
This time period represented the end of the technical assistance that was being offered to 
administrators, teachers, and students by NCREL and IIT. In addition, standardized testing 
occurred during this period and may have interfered with the implementation of data-driven 
decision making in the classrooms. Interviews with teachers suggested that these factors had a 
moderating effect on the implementation during this time. 

Math results from the ISAT were also analyzed for Grades 3 and 5 to compare the 
performance of students in the experimental and control classrooms. Overall, there was no 
significant difference in the average performance of the groups on this large-scale measure. 

Summary of Qualitative Results. As teachers shared charts of their student’s data, they learned 
to analyze the celeration lines, or learning trends reflected by the data, to guide their decisions. 
When data indicated that children were not learning (i.e., their data were “flat”), teachers and 
administrators explored instructional options, identified new teaching options and learning goals, 
and used the PT decision support system to reflect the efficacy of these interventions. 

During the fall semester, PT teachers were concerned that they were not “covering enough 
material” because their students were practicing skills until they became fluent and were working 
at a wide range of competency levels, while students in the control classes were moving through 
the curriculum at a predetermined pace. However, students responded very positively to the 
program’s timings, data plotting, and data sharing procedures. Students asked for timings, 
concentrated well, and negotiated individual learning goals with their teachers on a regular basis. 
As a result, teachers were able to accommodate the individual needs of their students because (a) 
they had data to reflect these needs, (b) they had learning materials that allowed for 
individualized practice and progress, and (c) the efficacy of instructional interventions was 
rapidly and efficiently manifested by student charts. 

In May 2000, teachers reported that their students continued to feel positively about the 
program. It was also reported that PT classes had ended up addressing more material than the 
control classes: they were able to make “leaps” in the curriculum without additional instruction. 
This progress was attributed to their focus on developing fluent basic math skills through this 



The Call for Data-Driven Decision Making in the Midwest’s Schools— 20 



implementation. Thus, their more efficient teaching process ended up saving weeks of time in 
spite of the investment in data collection, plotting, and analysis each day. 

These results are encouraging and suggest that maintaining and scaling up the data-driven 
decision making model over the course of a few years would accelerate student learning. With 
more efficient mastery of basic math skills, more attention could be devoted to enhancing 
student’s conceptual understanding of math as well as their problem-solving skills. These authors 
suggest that, in time, this would allow for improved performance on large-scale assessments that 
emphasize higher-order thinking skills, such as the ISAT. Finally, the study shows that the 
efficiency of learning can be raised in the area of math skills, and attempting to replicate these 
results in other academic areas should be a priority. 

SeeChange: PT Goes to the Web ('http://scc.iit.edu/beta) 

The results from implementing the “hands-on,” PT-based decision support system in Creve 
Coeur were indeed promising. Yet the history of PT and feedback from the Creve Coeur teachers 
suggested that scaling up this strategy would be limited by the need to maintain data and 
individual student charts via paper and pencil. “SeeChange,” a Web-based tool, was conceived to 
overcome this obstacle. 

Codeveloped by HT and NCREL, SeeChange allows these strategies to be streamlined via 
online data entry, archiving, and graphing. Using the site, educators who measure student 
performance each day have a rapid, user-friendly way to analyze and share these data for decision 
making (See Figure 4). 

SeeChange allows each user, using a password, to enter, archive, access, and aggregate 
student performance data on a 24/7 basis via the Internet. “Permission levels,” based on the role 
and appropriate data needs of various users, are assigned to protect student confidentiality. The 
site is designed to grow over time as users archive a wealth of data on learning indicators (e.g., 
state learning standards) in different subject areas and use these data to evaluate and improve 
outcomes of interventions, teaching practices, and curriculum. 

The SeeChange Web site also allows all educational stakeholders (e.g., students, teachers, 
administrators, and parents) to view student progress on an ongoing basis. Administrators can 
“see change” and rearrange administrative structure to help teachers work with learners to induce 
change. Finally, frequent and specific feedback empowers learners to improve. 
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Figure 4. SeeChange illustration 




Evaluation. SeeChange will be piloted and formatively evaluated during the next two years. 
The objective of this work will be to help NCREL/11T understand how SeeChange is used by 
various stakeholders and the impact it has on student learning. This feedback, in turn, will inform 
future development of the site. 

Curriculum Mapping With TLNs: Intersection of Program 
and Demographic Data 

The recent national curriculum reform movement stresses the creation of rigorous academic 
standards. Among the criteria outlined by the American Federation of Teachers (AFT) is the 
assertion that standards must “reflect various levels of knowledge and skills comparable to what 
students in high-achieving countries are expected to master” (Gandal, 1995). Researchers 
responded with an analysis of curriculum data, a form of program data, generated by the General 
Topic Tracing Map (GTTM), a simple and versatile data collection tool used in the Third 
International Math and Science Study (TIMSS). GTTM data created an overall picture of the 
“composite U.S. curriculum” in math and science from K-12 and enabled international 
comparisons. Findings indicated that the U.S. mathematics curriculum is unfocused and lacking 
in rigor and that, overall, the U.S. curriculum is designed to cover far more topics than the 
international average (Schmidt, McKnight, Valverde, Houang, & Wiley, 1997; Schmidt, 
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McKnight, & Raizen, 1997). By maintaining a high number of topics, each topic is less likely to 
be covered in detail. The result is a U.S. curriculum of great breath, but lacking in depth. 

Although enlightening, these findings provide little information for individual school districts 
given the existing variance in curriculum guides and textbooks. For local educational systems to 
examine and reflect upon the appropriateness of their own curricula, they need data that: 

■ Provide an overview of the breadth of the curricula. 

■ Illustrate the flow of topics through the curricula. 

■ Indicate the duration of time topics are in the curricula. 

■ Signify the rigor of the curricula. 

The task of generating data about these facets of curricula is currently time and labor 
intensive for schools, and avenues to relate these data to demographic variables or to a broader 
national or international context are few. This limits the ability of schools to objectively analyze 
their curricula along these domains. It also suggests that there are few resources for schools 
aiming to improve and optimize the design of their curricula. 

The GTTM provides a potential solution to this quandary by generating data that reflect 
characteristics of local curricula across these four domains. Because GTTM is tied to TIMSS, 
parallel information about curricula breadth, flow, duration, and rigor is available for the U.S. and 
for over 40 nations. The results of these analyses can be used to make comparisons across the 
curricula, to link curricular trends to demographic variables, and to guide data-driven reform. 

Bringing Curriculum Mapping to the Field 

In 1997, NCREL formed a partnership with the First in the World Consortium (FITW), 
consisting first of 10, and eventually 20 school districts north of Chicago. Participating FITW 
superintendents had made a collective agreement to pursue the national goal that U.S. students 
would become “first in the world” in mathematics and science achievement. An important goal of 
the FITW consortium was to discern if their curricula were “a mile wide and an inch deep,” as 
was the composite U.S. curriculum. The consortium successfully petitioned the U.S. Department 
of Education for the right to assess its students using TIMSS. It received a multitude of data as a 
result, but had little direction regarding how to proceed. NCREL provided technical assistance for 
analyzing the TIMSS data. NCREL also assisted in administering the GTTM to all participating 
districts. In partnering with FITW, NCREL aimed to better understand the specific needs of 
practitioners as they used large-scale assessment and survey data to guide local school 
improvement processes. 
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Considerable NCREL support was required to make GTTM data “user friendly” at the 
consortium, district, and school level. In addition to processing these data, NCREL compiled 
consortium, national, and international GTTM data into a report, 10 along with guidelines for 
conducting comparative analyses. NCREL also conducted several training sessions to show 
FITW stakeholders how to use the report and the data within. Finally, NCREL facilitated the 
analysis of GTTM data by a group of local practitioners. 

Scaling Up Curriculum Mapping 

During 2000 NCREL began to scale up the GTTM along two avenues. First, a self- 
administered paper and pencil workbook was produced, complete with generic displays for the 
user to plot data. Second, a Curriculum Mapping Web site was created, a more sophisticated 
Web-based version allowing users to enter GTTM data online, call up multiple displays with their 
data, and superimpose these data against data representing the U.S. and other TIMSS countries." 
The Web site was conceived as a simple, fast, and accessible tool that would allow districts to 
reform local curricula through data-driven decisions. The workbooks were conceived as both 
stand-alone tools for basic analyses as well as a “teaser” for the Web site. 

Several issues came to a head to lead NCREL to embark on these scaling-up efforts. First, the 
national curriculum reform movement, instigated by the TIMSS, motivated local districts to 
reflect upon the breadth and depth of their curricula. Second, NCREL’s experience with FITW 
indicated a dependence on technical assistance by other local practitioners and administrators 
interested in doing curriculum mapping. Finally, NCREL anticipated additional needs from new 
benchmark groups, including five consortia and 13 states that had participated in the TIMSS-R 
(repeat). A number of these benchmark groups reside in the NCREL region. 12 These groups now 
have access to their own rich survey data from the TIMSS-R. But the GTTM, administered with 
the TIMSS, was not administered with the TIMSS-R. For these groups to take full advantage of 
their participation in the TIMSS-R, they would need access to the GTTM as well as assistance 
with the resulting data. 

Beginning in spring of 2000, NCREL simultaneously developed the workbooks and the Web 
site in partnership with the U.S. TIMSS National Research Center at Michigan State University, 
the major publishers of U.S. and international GTTM data from the TIMSS. Construction of the 
Web site was contracted to the Center for the Advancement of Information Technology (CAIT) at 
Western Illinois University. Both tools were completed in December 2000. 
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NCREL has become nationally recognized through conference presentations, papers, and 
positive word-of-mouth from FITW for its expertise in working with TIMSS data at the 
practitioner level. NCREL staff have been invited to conduct interactive workshops on the 
Curriculum Mapping Web site at two professional conferences (National Staff Development 
Council, December 2000; Association for Supervision and Curriculum Development, March 
2001). In addition, a brief mention of the Curriculum Mapping Web site on a TIMSS e-mail list 
(6/1/00) quickly generated a great deal of interest. Within weeks, approximately 20 contacts, from 
Alaska to Florida, inquired about the roll-out date of the site and future NCREL products related 
to the TIMSS and/or GTTM. 

A DSS for Curriculum Mapping (www.ncrel.org/currmap ) 

In the next few years, NCREL will continue to partner with the U.S. TIMSS National 
Research Center to use and expand the curriculum mapping Web site. This partnership will 
include systematically collecting GTTM data from TIMSS-R benchmark groups through the Web 
site, then describing the curriculum profiles reported by benchmark groups, and national and 
international Web site users. These data will also help NCREL build upon our preliminary 
findings regarding demographic variables and curriculum topic coverage. Specifically, NCREL 
will use the curriculum mapping Web site data to identify schools and districts throughout the 
nation with at-risk populations to research the relationship between student demographics (e.g., 
ethnicity, socioeconomic status) and curricular rigor. 

Two avenues of expansion will soon begin. Early in 2001, NCREL will begin field research 
to expand the curriculum mapping Web site to include Grades 9 through 12. NCREL will also 
work with the U.S. TIMSS National Research Center to include mappings of state standards for 
most of the United States. This will significantly enhance districts’ ability to align local curricula 
to state standards. 

Evaluation. The curriculum mapping Web site will be piloted and formatively evaluated 
during the next contract period. The objective of this work will be to help NCREL and the U.S. 
TIMSS National Research Center understand how various stakeholders use the site and the 
impact it has on decision making regarding curriculum design. This feedback, in turn, will inform 
future development of the site. 
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Conclusion 



As a result of our research, tool development, and dissemination to the field in various 
formats (e.g., conference presentations, articles), NCREL has acquired a reputation for significant 
expertise with data-driven decision making and education decision support systems. At this point, 
there is some consensus among researchers and policymakers regarding the need to use data 
better in schools. The accountability movement has apparently pressed this issue to the point of 
recognition, if not acceptance, by practitioners as well. Therefore, questions regarding “why” data 
should be used for educational decision making have been somewhat quelled. The lingering 
question is “How?” That is, how can practitioners come to make better decisions by accessing, 
analyzing, and using relevant data? Significant barriers exist, such as lack of training and time, 
existence of competing demands, and resistance to change, that inhibit the capacity of educators 
to move from problem recognition to the process of changing and improving practices. 

NCREL may appear to be “ahead of the curve” in this area of educational research and 
development due to the high visibility and initial success of the DSSs we have helped to develop, 
but we remain acutely aware of the need for additional knowledge regarding issues such as: 

• If, how, and by whom decision support systems are used 

• The relationship between the use of DSSs and the style of decision making in schools 

• Identifying effective professional development to complement launches of online 
DSSs 

• Understanding the limitations of DSSs for particular users and specific types of 
decision making. 

The development of DSSs in education is emerging and dynamic, fueled by the movements 
of accountability, assessment, technology, and management information systems. During the next 
5-years, NCREL will work to respond to the above issues, and will consult, collaborate, and 
network in this area while considering partnerships that may benefit our constituents and help us 
pinpoint our contribution to the field. 

NCREL’s new signature area, Education Decision Support Systems (EDSS), has emerged 
from the work reported here and aims to make data more available and useful for educators. It 
will be important for this signature area to keep pace with the changing needs and capacities of 
educators and information systems, and to fine-tune the specifics of its overall objectives 
accordingly. One thing is clear: Educators and policymakers will continue to call for support in 
their efforts to improve the education system through informed decision making. Over the next 
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contract period, it will be important that we at NCREL expand our knowledge, link our work to 
others, apply, evaluate, and share our findings. 



1 SSAP continues to be updated annually, and is now maintained by the CCSSO alone. It can be accessed 
from the CCSSO Web site, http://www.ccsso.org. 

2 This representation has been adapted from a model developed by Judy Sargent, Ph.D., for NCREL’s Data 
Retreat Participant’s Guide and Data Retreat Facilitator’s Guide , 2000. 

3 This year (2000), Evaluation Software Publishing, Inc. (ESP) and the Council of Chief State School 
Officers (CCSSO) worked together on behalf of the Office of the Chief Information Officer, U.S. 
Department of Education (OCIO) to host and summarize findings from several regional and national 
meetings addressing the status of decision support systems in education. Much of the contextual 
information offered here has been gleaned, with permission, from summary reports that are available at: 
www.EducationAdvisor.com. 

4 More details about the development process of ILSI can be found in Appendix 1. 

5 Further information at http://qsp.cse.ucla.edu. 

6 More details about the development process of WINSS can be found in Appendix 2. 

7 These data were collected by the Web trackers on the WINSS and ILSI sites. 

8 For a more comprehensive description, see Characteristics of Current Decision Support Systems in 
Education (ESP, 2000), available at: 

http://www.educationadvisor.com/ocio2000/4i Characteristics of Current DSSs.doc 

9 A detailed evaluation report is beyond the scope of this document. Instead, a summary of the evaluation 
goals, activities, and results are provided. 

10 Kroeze, Masini, & Aumiller. (2000). GTTM Report: Mathematics and Science Curricula. North Central 
Regional Educational Laboratory, Oak Brook, IL. 

11 The Web site will also provide for NCREL a database of the curriculum and demographic data entered 
by users. 

12 Illinois, Indiana, Michigan, Ohio, Chicago Public Schools (Illinois), Naperville School District (Illinois), 
SMART Consortium (Ohio), and Connected Mathematics Project (Michigan). 
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APPENDIX 1 



Development Chronology for 

Illinois School Improvement Web site (ILSI) and Customized QSP for IL 


6/99 


ISBE requests help to develop an “electronic reporting” DSS for state and local 
school data 


6/99 


NCREL proposes codevelopment partnership with ISBE and the Illinois Business 
Roundtable (IBR) 


7/99 


Initial partnership meeting _____ 


9/99 


Matrix of responsibilities for ELSI and QSP project activities developed and 
disseminated to NCREL/ILBE/1BR partners 


10/99 


IBR hosts meeting with state educators to introduce plans for DSS development 
linked to Illinois Learning Standards 


11/99 


NCREL hosts pilot group of educators for School Improvement Planning Project, 
along with ISBE and IBR. 

• Project Overview 

• Demonstration of Maryland DSS and QSP 

• DSS development for EL 


12/99 


ISBE staff meeting with State Superintendent, Max McGee, Associate 
Superintendent, Bill Conrad, and NCREL Executive Director, Gina Burkhardt 

• ISBE and NCREL commitment to project 

• Progress of project 

• Subcontractors on project 

• Organizational responsibilities for next steps 

■/ Training 
■/ Site maintenance 
•S Site content 
•f Data sets 
•/ Data Analyses 


Pilot group of educators for School Improvement Planning Project meets in 
Lexington, Illinois, along with NCREL, ISBE and IBR. 

• Demonstration of EL DSS framework 

• Pilot School feedback 

• Subgroup meetings 

✓ Technical 

✓ Site design 
Piloting/evaluation 


1/00 


ELSI password-protected Alpha Site posted . . 


IBR Presentation of Illinois School Improvement Web site to ISBE and EL 
Business Leaders’ Education Summit. 



2/00 


Plans for Illinois School Improvement (ELSI) Web site pilot plan 
S Field test Web site DSS 

■S Field test QSP tool with help from graduate students from Governor’s State 
University 

y Pilot school uses ILSI and QSP on own for several weeks 
■/ Focus group meeting with pilot schools to evaluate success of these projects 
in helping schools review their school improvement plans 




Results used to create ILSI Beta Web site and customized QSP software 




Planning meeting: ISBE, NCREL, IBR 

• Updates: ILSI Web site and QSP piloting 

• Statewide rollout plans 
Fall staff development plans 


7/00 


ISBE hosts meeting with NCREL and EBR to discuss staff development plans for 
ILSI and QSP 

• Review of feedback from initial pilot groups 

• Training being delegated to Regional Offices of Education (ROE); ISBE to 
provide funding to 7 ROEs to form first training group with ILSI/QSP 

• ROE administrators will be aided by interns from Governor’s State University 




Hosting of site transferred from subcontractor server to ISBE server 


11/00 


Press release announcing official public rollout of ILSI Web site 
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Appendix 2 



Development Chronology for 

Wisconsin Information Network for Successful Schools (WINSS) 


1/00 


WINSS requests help to develop an “electronic reporting” DSS for state school 
data 




Initial partnership meeting: WT DPI and NCREL, Madison, Wisconsin 

• NCREL presentation of DSS templates 

• Roundtable discussion to define roles and responsibilities 

• Timeline 

• Deliverables 

• Subcontracts 

• List of data elements 


2/00 


WI DPI defines cross-department Web workgroup 


3/00 


Matrix of responsibilities for WINSS project activities developed and 
disseminated to NCREL/WI DPI partners 




WINSS password-protected Alpha Site posted 


6/00 


WI DPI, along with NCREL, hosts 3-day meeting with pilot groups to introduce, 
use, and gain feedback on the initial version of WINSS. Pilot groups included: 

• Policymakers (Governor’s office) 

• Administrators 

• Representatives from WI CESAs (Cooperative Educational Service Agencies) 

• Teachers 

• Community members 

• Parents 




Results used to create WINSS Beta Web site 


9/00 


Hosting of site transferred from subcontractor server to WINSS server 


9/00 


Press release announcing official public rollout of WINSS Web site 
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